222 research outputs found
Every Parameter Matters: Ensuring the Convergence of Federated Learning with Dynamic Heterogeneous Models Reduction
Cross-device Federated Learning (FL) faces significant challenges where
low-end clients that could potentially make unique contributions are excluded
from training large models due to their resource bottlenecks. Recent research
efforts have focused on model-heterogeneous FL, by extracting reduced-size
models from the global model and applying them to local clients accordingly.
Despite the empirical success, general theoretical guarantees of convergence on
this method remain an open question. This paper presents a unifying framework
for heterogeneous FL algorithms with online model extraction and provides a
general convergence analysis for the first time. In particular, we prove that
under certain sufficient conditions and for both IID and non-IID data, these
algorithms converge to a stationary point of standard FL for general smooth
cost functions. Moreover, we introduce the concept of minimum coverage index,
together with model reduction noise, which will determine the convergence of
heterogeneous federated learning, and therefore we advocate for a holistic
approach that considers both factors to enhance the efficiency of heterogeneous
federated learning.Comment: Accepted at NeurIPS 202
FedHAP: Federated Hashing with Global Prototypes for Cross-silo Retrieval
Deep hashing has been widely applied in large-scale data retrieval due to its
superior retrieval efficiency and low storage cost. However, data are often
scattered in data silos with privacy concerns, so performing centralized data
storage and retrieval is not always possible. Leveraging the concept of
federated learning (FL) to perform deep hashing is a recent research trend.
However, existing frameworks mostly rely on the aggregation of the local deep
hashing models, which are trained by performing similarity learning with local
skewed data only. Therefore, they cannot work well for non-IID clients in a
real federated environment. To overcome these challenges, we propose a novel
federated hashing framework that enables participating clients to jointly train
the shared deep hashing model by leveraging the prototypical hash codes for
each class. Globally, the transmission of global prototypes with only one
prototypical hash code per class will minimize the impact of communication cost
and privacy risk. Locally, the use of global prototypes are maximized by
jointly training a discriminator network and the local hashing network.
Extensive experiments on benchmark datasets are conducted to demonstrate that
our method can significantly improve the performance of the deep hashing model
in the federated environments with non-IID data distributions
Cooperative Multi-Type Multi-Agent Deep Reinforcement Learning for Resource Management in Space-Air-Ground Integrated Networks
The Space-Air-Ground Integrated Network (SAGIN), integrating heterogeneous
devices including low earth orbit (LEO) satellites, unmanned aerial vehicles
(UAVs), and ground users (GUs), holds significant promise for advancing smart
city applications. However, resource management of the SAGIN is a challenge
requiring urgent study in that inappropriate resource management will cause
poor data transmission, and hence affect the services in smart cities. In this
paper, we develop a comprehensive SAGIN system that encompasses five distinct
communication links and propose an efficient cooperative multi-type multi-agent
deep reinforcement learning (CMT-MARL) method to address the resource
management issue. The experimental results highlight the efficacy of the
proposed CMT-MARL, as evidenced by key performance indicators such as the
overall transmission rate and transmission success rate. These results
underscore the potential value and feasibility of future implementation of the
SAGIN
Federated PAC-Bayesian Learning on Non-IID data
Existing research has either adapted the Probably Approximately Correct (PAC)
Bayesian framework for federated learning (FL) or used information-theoretic
PAC-Bayesian bounds while introducing their theorems, but few considering the
non-IID challenges in FL. Our work presents the first non-vacuous federated
PAC-Bayesian bound tailored for non-IID local data. This bound assumes unique
prior knowledge for each client and variable aggregation weights. We also
introduce an objective function and an innovative Gibbs-based algorithm for the
optimization of the derived bound. The results are validated on real-world
datasets
Phylogenetic and evolutionary analysis of the septin protein family in metazoan
AbstractSeptins, a conserved family of cytoskeletal GTP-binding proteins, were presented in diverse eukaryotes. Here, a comprehensive phylogenetic and evolutionary analysis for septin proteins in metazoan was carried out. First, we demonstrated that all septin proteins in metazoan could be clustered into four subgroups, and the representative homologue of every subgroup was presented in the non-vertebrate chordate Ciona intestinalis, indicating that the emergence of the four septin subgroups should have occurred prior to divergence of vertebrates and invertebrates, and the expansion of the septin gene number in vertebrates was mainly by the duplication of pre-existing genes rather than by the appearance of new septin subgroup. Second, the direct orthologues of most human septins existed in zebrafish, which suggested that human septin gene repertoire was mainly formed by as far as before the split between fishes and land vertebrates. Third, we found that the evolutionary rate within septin family in mammalian lineage varies significantly, human SEPT1, SEPT 10, SEPT 12, and SEPT 14 displayed a relative elevated evolutionary rate compared with other septin members. Our data will provide new insights for the further function study of this protein family
Consistent Attack: Universal Adversarial Perturbation on Embodied Vision Navigation
Embodied agents in vision navigation coupled with deep neural networks have
attracted increasing attention. However, deep neural networks have been shown
vulnerable to malicious adversarial noises, which may potentially cause
catastrophic failures in Embodied Vision Navigation. Among different
adversarial noises, universal adversarial perturbations (UAP), i.e., a constant
image-agnostic perturbation applied on every input frame of the agent, play a
critical role in Embodied Vision Navigation since they are
computation-efficient and application-practical during the attack. However,
existing UAP methods ignore the system dynamics of Embodied Vision Navigation
and might be sub-optimal. In order to extend UAP to the sequential decision
setting, we formulate the disturbed environment under the universal noise
, as a -disturbed Markov Decision Process (-MDP). Based
on the formulation, we analyze the properties of -MDP and propose two
novel Consistent Attack methods, named Reward UAP and Trajectory UAP, for
attacking Embodied agents, which consider the dynamic of the MDP and calculate
universal noises by estimating the disturbed distribution and the disturbed Q
function. For various victim models, our Consistent Attack can cause a
significant drop in their performance in the PointGoal task in Habitat with
different datasets and different scenes. Extensive experimental results
indicate that there exist serious potential risks for applying Embodied Vision
Navigation methods to the real world
Towards Efficient Communications in Federated Learning: A Contemporary Survey
In the traditional distributed machine learning scenario, the user's private
data is transmitted between nodes and a central server, which results in great
potential privacy risks. In order to balance the issues of data privacy and
joint training of models, federated learning (FL) is proposed as a special
distributed machine learning with a privacy protection mechanism, which can
realize multi-party collaborative computing without revealing the original
data. However, in practice, FL faces many challenging communication problems.
This review aims to clarify the relationship between these communication
problems, and focus on systematically analyzing the research progress of FL
communication work from three perspectives: communication efficiency,
communication environment, and communication resource allocation. Firstly, we
sort out the current challenges existing in the communications of FL. Secondly,
we have compiled articles related to FL communications, and then describe the
development trend of the entire field guided by the logical relationship
between them. Finally, we point out the future research directions for
communications in FL
Understanding the Robustness of 3D Object Detection with Bird's-Eye-View Representations in Autonomous Driving
3D object detection is an essential perception task in autonomous driving to
understand the environments. The Bird's-Eye-View (BEV) representations have
significantly improved the performance of 3D detectors with camera inputs on
popular benchmarks. However, there still lacks a systematic understanding of
the robustness of these vision-dependent BEV models, which is closely related
to the safety of autonomous driving systems. In this paper, we evaluate the
natural and adversarial robustness of various representative models under
extensive settings, to fully understand their behaviors influenced by explicit
BEV features compared with those without BEV. In addition to the classic
settings, we propose a 3D consistent patch attack by applying adversarial
patches in the 3D space to guarantee the spatiotemporal consistency, which is
more realistic for the scenario of autonomous driving. With substantial
experiments, we draw several findings: 1) BEV models tend to be more stable
than previous methods under different natural conditions and common corruptions
due to the expressive spatial representations; 2) BEV models are more
vulnerable to adversarial noises, mainly caused by the redundant BEV features;
3) Camera-LiDAR fusion models have superior performance under different
settings with multi-modal inputs, but BEV fusion model is still vulnerable to
adversarial noises of both point cloud and image. These findings alert the
safety issue in the applications of BEV detectors and could facilitate the
development of more robust models.Comment: 8 pages, CVPR202
- …